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ABSTRACT 



This report presents findings of a review of the current 
research on the effectiveness of distance education in higher education. 

Major findings indicate, first, there is a paucity of truly original research 
dedicated to explaining or predicting phenomena related to distance learning; 
and, second, although most studies indicate that distance learning courses 
compare favorably with classroom-based instruction, the overall quality of 
the research is questionable and thereby renders the findings inconclusive. 
Key shortcomings of the research identified include: (1) much of the research 

does not control for extraneous variables and therefore cannot show cause and 
effect; (2) most of the studies do not use randomly selected subjects; (3) 
the validity and reliability of the instruments used to measure student 
outcomes and attitudes are questionable; and (4) many studies do not 
adequately control for the feelings and attitudes of students and faculty. 
Among specific research gaps identified are outcomes of complete programs 
rather than individual courses and reasons for high drop-out rates in 
distance courses. Implications drawn concern what "access to college" means, 
the importance of human contact in education, and the relatively limited 
importance of technology. Appended are an overview of the original research 
and other research syntheses. (Contains 70 references.) (DB) 
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Foreword 



M any states and institutions are experimenting with new forms of distance 
education— the electronic delivery of courses or entire academic programs by 
video and computer. States and individual colleges and universities are consid- 
ering whether to make major investments in this technology. In other words, they are 
implicitly deciding whether to maintain or to enlarge classroom-based academic programs 
or to substitute electronic programs, which they believe may offer a variety of economies. 

In the face of this, ordinary citizens wonder, as do educators, whether distance education 
may bar students from the crucial element of interchange with teachers, researchers, and 
other students that prepares students for a lifetime as knowledge workers. Other educa- 
tors contend that distance education is just like any other form of education; it can be 
done well or badly. 

As this report indicates, there is a good deal of research dealing with distance education. 
Proponents of distance education contend that the findings prove conclusively that dis- 
tance learning is ‘‘as good as” traditional education if conducted properly. But when we 
take a closer look, is this really so? Does distance education, for example, work better for 
some academic subjects than others? Does it work better for some students than others? 
Is there more of a dropout problem with distance education? Are library and information 
resources sufficient under distance education? What elements are necessary for student 
success and have they been evaluated? Does student assessment in distance education 
classrooms differ from that in traditional classrooms? Is the kind of knowledge acquired 
the same— particularly if more than a course or two, possibly the entire academic pro- 
gram, is delivered at a distance? Are there administrative issues affecting quality in dis- 
tance education? Are there shortcomings in the research itself? 
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The American Federation of Tfeachers and the National Education Association commis- 
sioned The Institute for Higher Education Policy to conduct a review of the current re- 
search on the effectiveness of distance education, to analyze what the research tells us 
and does not tell us. What this report suggests is that too many of the questions posed 
above are left unaddressed or unanswered in the research, while policymakers, faculty, 
and students need to make properly informed judgements about key issues in distance 
education. We hope this report will lead to further research and assist with policy develop- 
ment in this important area. 



Larry Gold 

Higher Education Director 
American Federation of Tfeachers 
www.aft.org 



Christine Maitland 
Higher Education Coordinator 
National Education Association 
www.nea.org/he 




Executive Summary 



D istance learning is not a new phenomenon. With the development of the postal 
service in the 19^^ century, commercial correspondence colleges provided dis- 
tance education to students across the country. This trend continued well into 
the 20^^ century with the advent of radio, television, and other media that allowed for 
learning at a distance. In the last decade, distance education has changed significantly 
with the use of computer-mediated learning, two-way interactive video, and a variety of 
other technologies. Colleges and universities are forging ahead to provide learning at a 
distance, and many institutions are making substantial investments in new technologies 
for teaching. 

What impact is all of this technology having on the educational effectiveness of colleges 
and universities? The amount of written material devoted to distance education is exten- 
sive, and includes policy papers, “how to" articles, and essays, as well as a limited, though 
not insignificant body of original research. With few exceptions, the bulk of these writings 
suggests that the learning outcomes of students using technology at a distance are similar 
to the learning outcomes of students who participate in conventional classroom instruc- 
tion. The attitudes and satisfaction of students using distance learning also are character- 
ized as generally positive. Thomas Russell, in his recently published annotated bibliogra- 
phy entitled The No Significant Difference Phenomenon, lists hundreds of sources that seem 
to bolster these arguments. However, a closer look at the evidence suggests a more cau- 
tious view of the effectiveness of distance education. 

The purpose of this analysis is to examine the research on distance learning more closely so 
that public policy may be better informed. What are the findings of the research on the 
effectiveness of distance education? Are they valid? Are there gaps in the research that 
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require further investigation and information? What does the literature suggest for the fu- 
ture? These questions are becoming more important as pressure grows—both from within 
and outside the academy— to use technology as a primary method for delivering higher 
education. Tb explore these issues in more detail, the American Federation of Tfeachers and 
the National Education Association commissioned The Institute for Higher Education Policy 
to conduct a review of the current research on distance education. Specifically, this report: 

► Reviews the findings of the original research and assesses the overall 
quality of the analysis; 

► Identifies gaps or omissions in the body of original research; and 

► Discusses the implications of the research. 

The scope of this review is limited to written material published during the 1990s, and 
particular attention is focused on those types of technologies currently being used by the 
majority of institutions. This paper concentrates primarily on an evaluation of the origi- 
nal research— including experimental, descriptive, correlational, and case studies— and 
summarizes the key information and findings of the other policy papers, articles, and 
essays that dominate the literature. While this review of original research does not en- 
compass every study published since 1990, it does capture the most important and salient 
of these works. 

What Does the Original Research Say About 
the Effectiveness of Distance Learning? 

It is important to emphasize that, despite the large volume of written material concentrat- 
ing on distance learning, there is a relative paucity of true, original research dedi- 
cated to explaining or predicting phenomena related to distance learning. From 
this more limited group of original research, three broad measures of the effectiveness of 
distance education are usually examined. These include: 

► Student outcomes, such as grades and test scores; 

► Student attitudes about learning through distance education; and 

► Overall student satisfaction toward distance learning. 
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Most of these studies conclude that, regardless of the technology used, distance learning 
courses compare favorably with classroom-based instruction and enjoy high student satis- 
faction. For example, many experimental studies indicate that students participating in 
distance learning courses perform as well as their counterparts in a traditional classroom 
setting. These studies suggest that the distance learning students have similar grades or 
test scores, or have the same attitudes toward the course. The descriptive analysis and 
case studies focus on student and faculty attitudes and perceptions of distance learning. 
The purpose of many of these types of research is to develop recommendations to im- 
prove distance learning. These studies typically conclude that students and faculty have a 
positive view toward distance learning. 

A closer look at the research, however, reveals that it may not be prudent to accept 
these findings at face value. Several problems with the conclusions reached through 
this research are apparent. The most significant problem is that the overall quality of 
the original research is questionable and thereby renders many of the findings 
inconclusive. 

The findings of the original research must be read with some caution. Assessing the qual- 
ity of the original research requires a determination that the studies adhered to com- 
monly accepted principles of good research. The analysis is much more than an academic 
exercise. These principles are essential if the results of the studies are to be considered 
valid and generalizable. If a study does not abide by these principles, the results can be 
erroneous or misleading, and therefore lead to conclusions that result in poor public policy. 

What Are the Key Shortcomings of the Research? 

The analysis has found that there are several key shortcomings of the original research on 
the effectiveness of distance learning. These include; 
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► Much of the research does not control for extraneous variables and therefore 
cannot shoiv cause and effect. 

Lack of control for extraneous variables is a major shortcoming that pertains particularly 
to experimental research, where the researcher attempts to compare the outcomes of a 
control group with the outcomes of an experimental group. Most experimental studies of 
distance learning are designed to measure how a specific technology— the “cause"— im- 
pacts upon some type of learning outcome or influences the attitudes of the course by 
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students-the "effect." Tb accurately assess this relationship, other potential "causes" must 
not influence the measured outcomes. In almost all of the experimental research reviewed 
for this paper, there was inadequate control of extraneous variables. 

► Most of the studies do not use randomly selected subjects. 

The single best way of controlling for extraneous variables is to assign students randomly 
to both the experimental and control groups. However, many of the published studies 
reviewed used intact groups for comparison purposes. As a result, these studies run the 
risk of having a number of variables affecting academic achievement or student satisfac- 
tion, not just the technology used to provide the education at a distance. 

► The validity and reliability of the instruments used to measure student out- 
comes and attitudes are questionable. 

An important component of good educational research relates to proper measurement of 
learning outcomes and/or student attitudes. In short, do the instruments-such as final 
examinations, quizzes, questionnaires, or attitude scales— measure what they are sup- 
posed to measure? A well-conducted study would include the validity and reliability of 
the instruments so that the reader can have confidence in the results. In almost all of the 
studies reviewed, this information was lacking. 

► Many studies do not adequately control for the feelings and attitudes of the 
students and faculty --what the educational research refers to as **reactive effects,^ 

Reactive effects are a number of factors associated with the way in which a study is con- 
ducted and the feelings and attitudes of the students involved. One reactive effect is known 
as the Novelty Effect, and refers to increased interest, motivation, or participation on the 
part of students simply because they are doing something different, not better per se. 
Another, called the John Henry Effect, refers to control groups or their teachers feeling 
threatened or challenged by being in competition with a new program or approach and, 
as a result, outdoing themselves and performing well beyond what would normally be 
expected. In many studies, precautions were not taken in the research to gu^rd against 
these effects. 
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What Are the Gaps in the Research That Require Further 
Investigation and Information? 

Notwithstanding the fact that the overall quality of the research needs improvement, there 
are several important issues regarding the effectiveness of distance learning that require 
further investigation and information. These gaps must be filled so that public policy dis- 
cussions are based on accurate and adequate information. Specific issues include: 

A. The research has tended to emphasize student outcomes for individual courses 
rather than for a total academic program. 

A major gap in the research is the lack of studies dedicated to measuring the effectiveness 
of total academic programs taught using distance learning. Virtually all of the compara- 
tive or descriptive studies focus upon individual courses. This raises important questions 
about whether a total academic program delivered by technology compares favorably 
with a program provided on campus. In addition to cognitive skills, and verbal, quantita- 
tive, and subject matter competence, outcomes with regard to critical thinking skills, atti- 
tudes and values, moral development, etc. need to be addressed. 

B. The research does not take into account differences among students. 

A substantial portion of research on distance learning has been conducted to demonstrate 
no significant difference in achievement levels between groups of distance and traditional 
learners. However, there is wide variance of achievement and attitudes within the groups, 
which indicates that learners have a variety of different characteristics. The factors influ- 
encing these differences could include gender, age, educational experience, motivation, 
and others. Gathering samples of students and amalgamating them into averages pro- 
duces an illusory "typical learner,'' which masks the enormous variability of the student 
population. Further research needs to focus on how individuals learn, rather than how 
groups learn. 

C. The research does not adequately explain why the drop~out rates of distance 
learners are higher. 

In a number of studies, there was evidence that a higher percentage of students partici- 
pating in a distance learning course tended to drop out before the course was completed 
compared to students in a conventional classroom. The issue of student persistence is 
troubling because of both the negative consequences associated with dropping out, and 
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the fact that the research could be excluding these dropouts— thereby tilting the student 
outcome findings toward those who are "successful." 

D. The research does not take into consideration how the different learning styles 
of students relate to the use of particular technologies. 

Understanding of how the learner, the learning task, and a particular technology interact 
is limited. Learner characteristics are a major factor in the achievement and satisfaction 
levels of the distance learner. Information regarding a student's preferred learning style 
will influence how the course is designed and the type of technology to be used. Addi- 
tional research could result in more information regarding why different technologies 
might be better suited for specific learning tasks. 

E. The research focuses mostly on the impact of individual technologies rather 
than on the interaction of multiple technologies. 

Much of the literature on distance learning focuses on one technology and either de- 
scribes its effectiveness and/or compares it to the conventional classroom experience. 
Most technologies, however, are multifunctional and can be adapted to address a wide 
range of learning outcomes. Unfortunately, there are few studies that examine more than 
one technology— and the synergistic effects of certain technologies— in addressing spe- 
cific educational outcomes and student groups. The few studies that are available do not 
provide ample grounds for generalization because of a range of limitations, including 
small sample sizes and lack of sufficient explanation of the instructional treatment. 

F. The research does not include a theoretical or conceptual framework. 

There is a vital need to develop a more integrated, coherent, and sophisticated program of 
research on distance learning that is based on theory. Theory allows researchers to build 
on the work of others and, therefore, increase the probability of addressing the more 
significant questions regarding distance learning. Using theory as a guiding framework 
also allows the research to be replicated and enhances its generalizability, making indi- 
vidual studies more meaningful. 

G. The research does not adequately address the effectiveness of digital Hihraries.^ 

Students participating in distance learning, particularly those in remote locations, are 
often introduced to a digital "library" that provides access to bibliographies, as well as full- 
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text, of a variety of resources. The library is at the core of the higher education experience 
and, especially at the graduate level, is an integral part of the teaching/learning process. 
Some digital libraries boast an enormous array of resources, with the implicit notion that 
they can provide the same service as the traditional library. But do digital libraries pro> 
vide adequate services for the academic programs they are established to support? Anec- 
dotal evidence seems to suggest that the curriculum objectives of some distance learning 
courses have been altered because of a limited variety of books and journals available 
from the digital library. 

Implications 

Research on distance learning has been driven by what many are calling the '‘information 
revolution." Advances in technology offer both the general public and faculty a dizzying 
array of challenges that are unprecedented. Technology is having, and will continue to 
have, a profound impact on colleges and universities in America and around the globe. 
Distance learning, which was once a poor and often unwelcome stepchild within the 
academic community, is becoming increasingly more visible as a part of the higher edu- 
cation family. But the research and literature reviewed for this paper indicate that the 
higher education community has a lot to learn regarding how, and in what ways, technol- 
ogy can enhance the teaching/learning process, particularly at a distance. 

There are at least three broad implications that can be derived from this review of the 
original research and the other literature. The first is that the notion of “access to college" in 
the distance learning context is unclear. Many of the advocates of distance learning tout 
access to college-level education as a raison d'etre for the proliferation of distance education. 
Indeed, in some states, public policy leaders are recommending using distance education in 
lieu of ‘T)ricks and mortar" learning. Of particular concern is access as it relates to the 
efficacy of computer-mediated learning. Unlike two-way interactive video, where students 
and the instructor can see and talk to each other in a conventional classroom, computer- 
mediated learning requires special skills of students and more sophisticated technical sup- 
port if students are to interact fully. Questions that need to be asked include: What is the 
“quality" of the access? Does the student have the necessary skills to use the technology? 
What are the best ways to participate in asynchronous communication? Is there adequate 
technical support? Perhaps most important, will the cost of purchasing a computer and 
maintaining software be prohibitive for a substantial number of students? 
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Second, it seems clear that technology cannot replace the human factor in higher educa- 
tion. Faculty involved in distance education find themselves being a combination of content 
experts, learning process design experts, process implementation managers, motivators, 
mentors, and interpreters. In short, technology “can leverage faculty time, but it cannot 
replace most human contact without significant quality losses," as one expert has stated. 

Third, although the ostensible purpose of much of the research is to ascertain how tech- 
nology affects student learning and student satisfaction, many of the results seem to indi- 
cate that technology is not nearly as important as other factors, such as learning tasks, 
learner characteristics, student motivation, and the instructor. The irony is that the bulk 
of the research on technology ends up addressing an activity that is fundamental to the 
academy, namely pedagogy— the art of teaching. Tb that extent, the research has had a 
salutary effect in that a rising tide lifts all boats. Any discussion about enhancing the 
teaching/learning process through technology also has the beneficial effect of improving 
how students are taught on campus. 

In a sense, the discussion has come full circle. The research on distance learning has a 
long way to go, and much of it is inconclusive. On the other hand, technology has helped 
the academy to continue its focus on the essential goals of teaching and learning. As a 
result, either implicitly or explicitly, the key question that needs to be asked is: What is 
the best way to teach students? 
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Introduction 



D istance learning is not a new phenomenon. With the development of the postal 
service in the 19^^ century, commercial correspondence colleges provided dis- 
tance education to students across the country. This trend continued well into 
the 20**^ century with the advent of radio, television, and other media that allowed for 
learning at a distance. In the last decade, providing education at a distance has changed 
significantly as the use of computer-mediated learning, two-way interactive video, and 
other technologies has increased. Many universities are making substantial investments 
in new technologies for teaching. 

What impact is all of this technology having on the educational effectiveness of colleges 
and universities? William Wulf, professor of engineering and applied science at the Uni- 
versity of Virginia, makes the following observation about this issue: 

"Universities are in the information business, and technological develop- 
ments are transforming that industry. University professors from a vari- 
ety of disciplines have helped create the technologies that are forcing 
many U.S. industries to reinvent themselves, have advised industry lead- 
ers on how to adapt, and have analyzed the importance of changes for 
society. But it is harder to look inward to the university and to think about 
whether it might change in dramatic ways.”^ 

In recent years, colleges and universities have spent a great deal of time and effort in look- 
ing inward to analyze the efficacy of learning through the use of technology. This includes 
policy papers, "how to” articles, and essays, as well as a limited, though not insignificant 
body of original research. With few exceptions, the bulk of these writings suggests that the 
learning outcomes of students using technology at a distance are similar to the learning 
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outcomes of students who participate in conventional classroom instruction. The attitudes 
and satisfaction of students using distance learning also are characterized as generally posi- 
tive. Indeed, Thomas Russell's recently published annotated bibliogra- 
phy of papers, articles, and research studies about the effectiveness of 
distance learning, entitled The No Significant Difference Phenomenon, 
lists 355 sources dating back as early as 1928 that seem to bolster these 
arguments.^ However, a closer look at the evidence suggests a more 
cautious view of the effectiveness of distance education. 

There is no question that there is an impressive amount of writing that 
concludes that distance learning is viable and effective. The purpose 
of this analysis is to examine the research more closely so that public 
policy maybe better informed. What are the findings of the research on the effectiveness of 
distance education? Are they valid? Are there gaps in the research that require further 
investigation and information? What does the literature suggest for the future? These ques- 
tions are becoming more and more important as pressure grows— both from within and 
outside the academy— to use technology as a primary method for delivering higher educa- 
tion. lb explore these issues in more detail, the American Federation of Tfeachers and the 
National Education Association commissioned The Institute for Higher Education Policy to 
conduct a review of the research on distance education. 

This report is designed to take an objective look at the research to help answer these 
questions. Specifically, the report: 

► Reviews the findings of the original research and assesses the overall 
quality of the analysis; 

► Identifies gaps or omissions in the body of original research; and 

► Discusses the implications of the research. 

In addition, the appendices include a description of the range and variety of studies con- 
ducted, along with a summary of the content of articles found in major distance educa- 
tion publications, the vast majority of which are not original research studies. 



In the last decade, providing 
education at a distance has 
changed significantly as the 
use of computer-mediated 
learning, two-way interactive 
video, and other technologies 
has increased. 
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Because of the large amount of information available, the scope of this review is limited to 
written material published during the 1990s. Particular attention is focused on those types 
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of technologies currently being used by the majority of institutions, as identified in a 1 997 
study by the National Center for Education Statistics (NCES). They include two-way inter- 
active video, one-way prerecorded video, two-way audio/one-way video, and computer- 
mediated learning.^ 



It is important to understand what is meant by "distance learning.” Because the technol- 
ogy is evolving, the definition of what distance learning is continues to change. Distance 
learning generally includes “synchronous communication,” which occurs when teacher 
and student are present at the same time during instruction— even if they are in two 
different places— and "asynchronous communication,” which occurs when students and 
teachers do not have person-to-person direct interaction at the same time or place. Ac- 
cording to a recent report by the Council for Higher Education Accreditation, distance 
learning also is characterized by several key factors. Among the most important are that 
the teaching/learning process involves activities where the learners are at a distance from 

the originator of the teaching material, and that a combination of 

media— including television, videotapes, audiotapes, videoconferencing, 
audioconferencing, e-mail, telephone, fax, Internet, computer soft- 
ware, and print— may be used.^ 



Analytic Approach 

While this review of original research does not encompass every study 
published since 1990, it does capture the most important and salient 
of these works. The analysis includes over 40 of these original works 
of research. In addition, several hundred articles, essays, and other 
writings published in major journals on distance learning have been collected. These 
other essays, narratives, and reviews of older research are helpful, but do not provide 
sufficient evidence to accurately assess the effectiveness of distance learning. This paper 
therefore concentrates primarily on an evaluation of the original research, and summa- 
rizes the key information and findings of the non-original research writings. 



What are the findings of the 
research on the effectiveness 
of distance education? Are 
they valid? Are there gaps in 
the research that require 
further investigation and 
information? 



The original research utilizes various approaches, including: descriptive research; case 
studies; correlational research; and experimental research.^ The overwhelming majority 
of research studies used one or more of these types of designs. (See page 12 for descrip- 
tions of each of these approaches.) 
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Research Approaches 

Descriptive research involves the collection of data to answer specific questions. Data are usually col- 
lected through questionnaires, interviews, or standardized attitude scales. An important component of 
descriptive research is the validation of the questionnaire in order to determine if it measures what it 
was developed to measure. Data also can be collected through observation, for instance, analyzing 
student grades. In distance learning, typical descriptive studies are concerned with the assessment of 
attitudes, opinions, and conditions. 

A case study is an in-depth investigation of one "unit." The researcher can use a variety of methods to 
gather data, however, the explanation of the unit is generally written in narrative form. In distance 
learning, the case study usually involves a class of students taking a course through some type of 
electronic means. 

Correlational research involves collecting data in order to determine whether, and to what degree, a 
relationship exists between two or more quantifiable variables. An estimate is provided of just how 
related two variables are. If two variables are highly related, a correlation coefficient near + 1.00 (or - 
1.00) will be obtained; if two variables are not related, a coefficient near .00 will be obtained. It is 
important to note that correlational research never establishes a cause-effect relationship. One ex- 
ample of a correlational study might be determining the relationship (correlation) between student 
satisfaction with an instructor and the type of technology used. 

Experimental research is the only type of research that can truly test hypotheses concerning cause- and- 
effect relationships. In an experimental study, the researcher manipulates at least one independent 
variable and observes the effect on one or more dependent variables. In other words, the researcher 
determines "who gets what," which group of subjects will get which treatment. The groups are gener- 
ally referred to as experimental and control groups. Ideally, in experimental research the groups to be 
studied are randomly formed before the experiment, a procedure not involved in other methods of 
research. The essence of experimentation is the notion of "control." The researcher strives to insure 
that the experiences of the groups are as equal as possible on all important variables except the inde- 
pendent variable. In a study of distance education, a good example is comparing one group of students 
receiving instruction in a traditional classroom (the control group) with another group of students 
receiving instruction through two-way interactive video (the experimental group). 




What Does the Original Research 
Say About the Effectiveness of 
Distance Learning? 



I t should be emphasized that the review provided striking evidence of the fact that 
there is a relative paucity of true, original research dedicated to explaining or pre- 
dieting phenomena related to distance learning. The more limited group of original 
research on the effectiveness of distance learning addresses a variety of issues. Three 
broad measures of the effectiveness of distance education are usually examined in the 
original research. These include: 

► Student outcomes, such as grades and test scores; 

► Student attitudes about learning through distance education; and 

► Overall student satisfaction toward distance learning. 

Most of these studies conclude that, regardless of the technology used, distance learning 
courses compare favorably with classroom-based instruction and enjoy high student satis- 
faction. Several examples illustrate this point. One study compared student grades and 
attitudes at the sending site for several two-way interactive video courses with student 
grades and attitudes at the receiving site. The courses included education, nursing, and 
philosophy. Students at the receiving site attained lower grades than the students at the 
sending site. On the other hand, receiving site students in general portrayed a more posi- 
tive attitude than sending site students.® Another study compared the results of a take- 
home essay exam for students who participated in a live broadcast televised graduate 
course in management of technology with the results for students in the on-campus class- 
room. The students participating in distance learning performed better than students in 
the conventional classroom and had less inter-student variation. Tbrm papers for the groups 
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also were compared and no significant difference was found. With respect to homework, 
the distance learning students performed at a higher level.'^ 

At the high school level, a study compared the attitudes and achievements scores of stu- 
dents participating in an anatomy and physiology course taught in a regular classroom and 
delivered through interactive satellite. No significant differences were found between the 
attitudes of either group toward the courses. However, the group taught via satellite had 
higher mean scores on an achievement test than students in the classroom.® 

In one study, students who registered for a Sociology 101 course on campus were told that 
the course instead would be taught by a one-way televised broadcast. The group's perfor- 
mance with respect to attrition and grades was then compared to another group of stu- 
dents from the community who took the same course, also via one-way televised broad- 
cast, but in their homes, not in a classroom on campus. Both sections experienced a high 
percentage of students who did not complete the course (44 per- 
cent of the on-campus students, and 33 percent of the "off-cam- 
pus" students). The off-campus students had much higher grades 
than the on-campus students, although both groups rated the course 
as good or excellent. A higher proportion of on-campus students 
reported that they would not recommend the telecourse to a friend.® 

Another study compared three groups of teachers participating in 
an in-service microcomputer applications training course: one 
group was taught by computer-mediated learning, another group 
of teachers was taught on campus, and yet another group was taught 
through correspondence. The outcomes measured included scores on achievement tests, 
time-on-task, student attitudes, and drop-out rates. While the results were mixed, the scores 
on achievement tests were highest for students taking a correspondence course and low- 
est for students participating in computer-mediated learning. The computer-mediated 
learning and correspondence students spent more time-on-task than the on-campus stu- 
dents. Furthermore, there was a significant difference in the attitudes of students; the 
computer-mediated learning group was less positive toward the course than the conven- 
tional classroom group. 



It should be emphasized that 
the review provided striking 
evidence of the fact that there 
is a relative paucity of true, 
original research dedicated to 
explaining or predicting 
phenomena related to 
distance learning. 




21 




What Does the Original Research Say About the Effectiveness of Distance Learning? 



15 



These examples of experimental research are consistent with many other studies that indi- 
cate students participating in distance learning courses perform as well as their counter- 
parts in a traditional classroom setting. As Russell notes, merely because the research sug- 
gests that there is no difference in student performance does not mean that distance learn- 
ing is necessarily better than other methods of learning, just that it can be as effective.” 

There is also a body of descriptive analysis and case studies that focuses on student and 
faculty attitudes and perceptions of distance learning. The purpose of many of these 
types of research is to develop recommendations to improve dis- 
tance learning. One such study examined the perceptions and 
attitudes of students and faculty toward computer-mediated learn- 
ing and courses broadcast over a local television channel. A ma- 
jority of faculty observed that the distance learning students were 
"more serious, accomplished, and articulate" compared to on-cam- 
pus students. The distance learning students also had stronger 
analytical skills and written communication skills, and were more 
"self-directed” than their on-campus counterparts. The students reported that they chose 
distance education because of the flexible schedule, and nearly half of the students 
noted that the instructional method also influenced their decision. Students preferred 
not to commute and "enjoyed the luxury of not having to commit to a specific class 
meeting time." Approximately 40 percent of the students reported missing the face-to- 
face interactions and one-fourth missed the group dynamics. One of the major recom- 
mendations of this study was to "explore alternative ways to meet students' interaction 
needs" on a continuous basis. 



These studies suggest that 
the distance learning students 
have similar grades or test 
scores, or have the same 
attitudes toward the course. 



Several conclusions can be drawn from research that examined courses offered to U.S. Army 
and Navy personnel through two-way interactive video. These conclusions include: the 
choice of technology influences the instructional design; origination and remote site tech- 
nicians are crucial; adequate training for staff is necessary; sufficient practice and technical 
support are necessary because personnel are often intimidated by new technology; and 
activities need to be designed to allow students to interact with each other. 



In another descriptive study involving two-way interactive video, graduate nursing stu- 
dents were surveyed about their satisfaction regarding the technology. The majority of 
the respondents were very positive concerning distance education and requested more 
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distance learning opportunities. There was, however, considerable dissatisfaction with 
the accessibility of the library. Graduate education students participated in another study 
to determine their perceptions of two-way interactive video courses. Using written sur- 
veys, interviews, and class observations, the authors reported very mixed reactions from 
the students, which resulted in several specific recommendations. In addition to recom- 
mending that the audio quality needed to be improved, it was suggested that future train- 
ing programs include information on successful strategies for instruction and use more 
graphics. A toll-free phone number was proposed to contact the instructors^ 

One final example from the original research is a study that attempted to identify specific 
student attributes associated with student success in computer-mediated learning. Stu- 
dent success was defined as whether newly enrolled students passed their first course 
using computer-mediated learning. The following student characteristics were identified 
as being correlated with success. 

► Students who rated themselves highly on various measures of persis- 
tence related to taking on new projects; 

► Married students; 

► Students who rated the consequences of not passing as serious; 

► Students who rated their chances of succeeding in their studies higher 
than non-completers; 

► Students who did not need support from others to complete difficult 
tasks and did not find it important to discuss course work with other 
students; 

► Students with high literacy levels; 

► Students who rated themselves as well organized in terms of time 
management skills and said they generally had the time to do what 
they intended to do; 

► Students who rated their formal and informal learning as high in terms 
of preparing them for university studies; and 

► Female students. 
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In sum, many experimental studies conclude that students participating in distance learning 
courses perform as well as their counterparts in a traditional classroom setting. These 
studies suggest that the distance learning students have similar grades or test scores, or 
have the same attitudes toward the course. The descriptive analysis and case studies fo- 
cus on student and faculty attitudes and perceptions of distance learning. The purpose of 
many of these types of research is to develop recommendations to improve distance learn- 
ing. These studies typically conclude that students and faculty have a positive view to- 
ward distance learning. A closer look at the research, however, reveals that it may not be 
prudent to accept these findings at face value. 



What Are the Key 
Shortcomings of the Research? 



R ussell's annotated bibliography of papers, articles, and research studies, entitled 
The No Significant Difference Phenomenon, is often cited as evidence of the fact 
that the very weight of the evidence renders the general conclusion of no signifi- 
cant difference to be almost indisputable. However, Russell's book and web site provide a 
window on the problems that can be encountered in pulling together such disparate infor- 
mation. For example, a closer examination of the range of docu- 
ments cited in the book reveals some concerns. One is that there is 
considerable cross-referencing, where many of the papers and sum- 
maries cite similar research and/or reference each other. This some- 
what inflates the number of documents that reach the no signifi- 
cant difference conclusion. In addition, many of the writings cited 
in the volume are not original research studies. There is, we be- 
lieve, no effort on the author's part to mislead readers; in fact, Russell 
makes it very clear that the book is a compendium of only those 
materials that present the case for no significant difference. Accordingly, his book has 
provided a valuable contribution to the literature on distance learning. However, the book 
does provide discerning readers with evidence that care needs to be exercised in evaluat- 
ing the original research and other documents and articles. 

In examining the more limited group of original research, several problems with the con- 
clusions reached through the research are apparent. The most important problem is that 
the overall quality of the original research is questionable and thereby renders many of 
the findings inconclusive. 



The most important problem 
is that the overall quality of 
the original research is 
questionable and thereby 
renders many of the 
findings inconclusive. 
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The findings of the original research must be read with some caution. Assessing the 
quality of the original research requires a determination that the studies adhered to 
commonly accepted principles of good research. The analysis is much more than an 

academic exercise. These principles are essential if the results of 

the studies are to be considered valid and generalizable. If a study 
does not abide by these principles, the results can be erroneous 
or misleading, and therefore lead to conclusions that result in 
poor public policy. 



In almost all of the 
experimental research reviewed 
for this paper, there was 
inadequate control of 
extraneous variables. 



The analysis revealed several methodological flaws that should give 
pause to an objective observer. This is not necessarily surprising. 

Merely being published in a journal or book, for example, does not guarantee the quality 
of the study or that it was reported accurately. For instance, in a study which involved a 
meta-analysis of interactive video instruction, 367 experimental or quasi-experimental 
studies initially were identified for consideration. However, only 63 studies (17 percent) 
were chosen for the analysis because the remaining studies had methodological flaws, 
such as small sample size, high subject mortality, or the lack of minimal data to compute 
effect size (including means, standard deviations, t test, and F values). 



Unfortunately, many of these same criticisms apply to the research reviewed for this 
paper. The following are the key shortcomings of the original research on the effective- 
ness of distance learning. 

Much of the research does not control for extraneous variables and therefore cannot 
show cause and effect. 

Lack of control for extraneous variables is a major shortcoming that pertains particularly 
to experimental research, where the researcher attempts to compare the outcomes of a 
control group with the outcomes of an experimental group. Most experimental studies of 
distance learning are designed to measure how a specific technology— the "cause"— im- 
pacts upon some type of learning outcome or influences the attitudes of students— the 
"effect." Ib accurately assess this relationship, other potential "causes" must not influence 
the measured outcomes. If other variables influence outcomes, it is impossible to attribute 
"cause" to the technology being used. 
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In almost all of the experimental research reviewed for this paper, there was inadequate 
control of extraneous variables. This analysis is consistent with a 1992 inquiry of 150 
articles and studies that compared distance education learning to conventional class- 
room instruction, which found that differences other than the technology “could not be 
ruled out as key causal agents."^® 

One example involves a study comparing students taking a statistics class in a traditional 
classroom and through computer-mediated learning. In the discussion section of the re- 
port, the author states the following: 

Therefore, from these data, I suspect much of the performance differ- 
ences can be attributed to student collaboration as to the technology, it- 
self In fact, the highest performing students (in both classes) reported 
the most peer interaction. Therefore, it is important that faculty contem- 
plating the use of the virtual format pay attention to the issue of real time 
collaboration, whether carried from within the traditional classroom or 
in the context of virtual space. This is the key variable that should be 
controlled in further research on the subject of virtual teaching. 

Most of the studies do not use randomly selected subjects. 

Lack of random selection of subjects is closely aligned with control of extraneous vari- 
ables, and is another major weakness in the research evaluating distance learning. The 
single best way of controlling for extraneous variables is to assign students randomly to 
both the experimental and control groups. Using the previous example, if random selec- 
tion of subjects was used, it would be expected that the students would be similar with 
respect to motivation, experience, age, etc. However, many of the published studies re- 
viewed used intact groups for comparison purposes. As a result, these studies run the risk 
of having a number of variables affecting academic achievement or student satisfaction, 
not just the technology used to provide education at a distance. 

One important caveat: random selection does not guarantee that the control and experi- 
mental groups are similar in all attributes, but it is an accepted practice in good research 
and is considerably better than using intact groups. In addition, it should be noted that 
while there are statistical procedures for pretesting that can be utilized if intact groups are 
used, in many studies these procedures were not employed. 



ERIC 



27 




What Are the Key Shortcomings of the Research? 




The validity and reliability of the instruments used to measure student outcomes and 
attitudes are questionable. 



An important component of good educational research relates to proper measurement of 
learning outcomes and/or student attitudes. In short, do the instruments— such as final 
examinations, quizzes, questionnaires, or attitude scales— measure what they are sup- 
posed to measure? A well-conducted study would include the validity and reliability of 
the instruments so that the reader can have confidence in the results. In almost all of the 

studies reviewed, this information was lacking. Not only were the 

data regarding the appropriateness of the instruments absent, it 
was clear that the validity and reliability of the tests or question- 
naires were not even determined. 



In a study assessing students' initial perceptions of distance learn- 
ing, the author developed a survey based upon other surveys tar- 
geting course satisfaction. There was no evidence that the survey 
used pilot testing or other techniques to enhance validity.^® Many 
other studies involved the use of teacher produced examinations, 

which have not followed established methods for ensuring high levels of validity and 
reliability. It is rare to find a teacher-made test in the research that is based upon persua- 
sive evidence of content or construct validity. 



Many of the published studies 
...run the risk of having a 
number of variables affecting 
academic achievement or 
student satisfaction, not just 
the technology used to provide 
education at a distance. 



Many studies do not adequately control for the feelings and attitudes of the students 
and faculty— what the educational research refers to as "reactive effects." 

Reactive effects are a number of factors associated with the way in which a study is 
conducted and the feelings and attitudes of the students involved. TWo are worth men- 
tioning here because they were evident in many of the studies under review. One reac- 
tive effect, known as the Novelty Effect, refers to increased interest, motivation, or par- 
ticipation on the part of students simply because they are doing something different, 
not better per S6. Another, called the John Henry Effect, refers to control groups or their 
teachers feeling threatened or challenged by being in competition with a new program 
or approach and, as a result, outdoing themselves and performing well beyond what 
normally would be expected. When this effect occurs, the experimental treatment may 
not appear to be very effective. 
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The'Novelty Effect in particular can be observed when distance learning is used for the 
first time. In one case study, the first offering of a distance learning course was investi- 
gated. Using a variety of observational methods, the attitudes of both students and faculty 
regarding the delivery mode and methods of instruction were ascertained. Since the stu- 
dents and faculty were keenly aware of this new venture, their responses needed to be 
analyzed with some level of reservation.^^ 

Summary 

A significant portion of the research reviewed possesses serious weaknesses that call into 
question their results. That is not to say that well-constructed research designs were not 
found in the literature. Indeed, studies were found that provided the reader with some 
confidence in the findings. Unfortunately, they were in the minority. 

Moore and his colleagues make this point plainly in their review of the literature on 
distance learning's effectiveness:^^ 

The weight of evidence that can be gathered from the literature points 
overwhelmingly to the conclusion that teaching and studying at a dis- 
tance, especially that which uses interactive electronic telecommunica- 
tions media, is effective, when effectiveness is measured by the achieve- 
ment of learning, by the attitudes of students and teachers, and by cost 
effectiveness. 

Tb this optimistic conclusion it is necessary to append several reserva- 
tions. The sheer weight of opinion in the literature should not be taken as 
conclusive of itself, since most of it is based on anecdotal evidence of- 
fered by persons and institutions with vested interests in the techniques 
being evaluated, or in the very programs they are evaluating. Further- 
more, in those studies where some attempt has been made to gather em- 
pirical data, the research has been undertaken by schoolteachers or uni- 
versity faculty with extremely limited resources. As a result, the method- 
ology of many of the research designs is weak, with regard to such factors 
as the populations being compared or otherwise studied; the treatments 
being given, the statistical techniques being applied, and the validity, re- 
liability, and generalizability of the data on which the conclusions are 
based. 





What Are the Gaps in the 
Research That Require Further 
Investigation and Information? 



N otwithstanding the fact that the overall quality of the research needs improve- 
ment, there are several important issues regarding the effectiveness of distance 
learning that require further investigation and information. These gaps must be 
filled so that public policy discussions are based on accurate and adequate information. 
The discussion below identifies those areas of research that either need to be strength- 
ened or are absent from the literature. 

The research has tended to emphasize student outcomes for individual courses rather 
than for a total academic program. 

A major gap in the research is the lack of studies dedicated to measuring the effectiveness 
of total academic programs taught using distance learning. Virtually all of the compara- 
tive or descriptive studies focus upon individual courses. This raises important questions 
about whether a total program delivered by technology compares 
favorably with a program provided on campus. Many institutions 
advertise that students can complete a degree without leaving their 
home by participating in distance learning in a number of ways, 
including computer-mediated learning, receiving videotapes, or 
watching courses through their local cable network. 

The overall impact of a college education, particularly the bacca- 
laureate degree, is of considerable concern to educators, the gen- 
eral public, and public policy leaders alike. A college education is intended to impart 
much more than an accumulation of facts. The seminal study How College Affects Students, 
by Pascarella and Threnzini, provides an excellent summary of the outcomes of an under- 
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and amalgamating them into 
averages produces an illusory 
"typical learner," which masks 
the enormous variability of 
the student population. 
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graduate degree.^^ In addition to certain cognitive skills and verbal, quantitative, and sub- 
ject matter competence, the authors identify several other outcomes of students who 
complete a college degree. They include psychosocial changes, such as identity, self-con- 
cept, self-esteem, and relating to others in the external world; advancement of critical 
thinking skills; development of attitudes and values; moral development; and career choice 
and development. 

Tb the extent that these attributes are important, an entire body of research needs to be 
developed to determine if students participating in distance learning for their whole pro- 
gram compare favorably with students taught in the conventional classroom. Tb be sure, 
researchers tend to measure what they can measure. Conducting such research would 
take an enormous commitment of both time and money, let alone finding an adequate 
sample. On the other hand, this issue strikes at the very core of what a college education 
is all about and the degree to which distance education students benefit. 

The research does not take into account differences among students. 

A substantial portion of research on distance learning has been conducted to demon- 
strate no significant difference in achievement levels between groups of distance and 
traditional learners. However, there is wide variance of achievement and attitudes within 
the groups, which indicates that learners have a variety of different characteristics. The 
factors influencing these differences could include gender, age, educational experience, 
motivation, and others. Gathering samples of students and amalgamating them into av- 
erages produces an illusory "typical learner," which masks the enormous variability of 
the student population.^" 

Stated more plainly, experimental studies in distance learning are using an "agricul- 
tural-botany paradigm— assuming that students react to different educational treatments 
as consistently as plants react to fertilizers. Further research needs to focus on how 
individuals learn, rather than how groups learn. This is particularly important because 
technology has the potential to individualize learning to a greater degree than previ- 
ously known. 
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The research does not adequately explain why the drop-out rates of distance learners 
are higher. 

In a number of studies, there was evidence that a higher percentage of students partici- 
pating in a distance learning course tended to drop out before the course was completed 
compared to students in a conventional classroom. For instance, in one study, while the 
mean pass mark was 81 percent, only 40 percent of the students successfully completed 
the course.^® In another study, one-third of the students in a videoconferencing class re- 
ceived the grade of "I" (incomplete), compared to only 15 percent in an on-campus course.^^ 
In another study comparing an engineering course taught in a conventional classroom to 
one taught through computer-mediated learning, 95 percent of the 
resident students finished the course, compared to 64 percent of 
the computer-mediated learning students.^® One other study found 
that students participating in computer-mediated learning had sig- 
nificantly higher incompletion rates (32 percent) than the on-cam- 
pus students (4 percent). 

The issue of student persistence (called subject mortality by re- 
searchers) is troubling because of both the negative consequences 
associated with dropping out, and the fact that the research could 
be excluding these dropouts— thereby tilting the student outcome findings toward those 
who are "successful." For example, if the average mean score of an achievement test or 
the score from a survey measuring student satisfaction is based upon a smaller number of 
students than had registered for the course, the data could be invalid. In other words, if 
the students who had dropped out had remained in the course, the mean score of the 
achievement test or student survey could be significantly different. 

A major reason for providing distance education is to provide access to students who 
normally would not be able to participate in higher education. If a substantial number of 
students fail to complete their courses, the notion of access becomes meaningless. Dis- 
tance learning studies should clearly indicate the beginning and ending number of stu- 
dents in a course to ensure that dropouts are not excluded from the analysis. 



There are few studies that 
examine more than one 
technology— and the synergistic 
effects of certain 
technologies — in addressing 
specific educational outcomes 
and particular student groups. 
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The research does not take into consideration how the different learning styles of 
students relate to the use of particular technologies. 

Similar to the relationship of course content and technology, understanding of how the 
learner, the learning task, and a particular technology interact is limited. Learner charac- 
teristics are a major factor in the achievement and satisfaction levels of the distance learner. 
Information regarding a student's preferred learning style will influence how the course 
is designed and the type of technology to be used. The development of software could be 
enhanced from a better understanding of the properties of technology (such as interactive 
and passive characteristics) with factors such as complexities of tasks, and cognitive, af- 
fective, and psychomotor skills.^° Additional research could result in more information 
regarding why different technologies might be better suited for specific learning tasks. 

In one study that attempted to address these issues, students were self-assigned to two 
groups: an on-campus conventional class, and a remote distance learning class using two- 
way interactive video. The students were administered the Canfield Learning Styles In- 
ventory to ascertain each student's primary learning style. The author concluded that 
learning style preferences may affect academic achievement and attitudes of students 
participating in distance learning. However, the study has limited validity because of a 
small number of subjects, all of which were female. 

The research focuses mostly on the impact of individual technologies rather than on 
the interaction of multiple technologies. 

Much of the literature on distance learning focuses on one technology and either de- 
scribes its effectiveness and/or compares it to the conventional classroom experience. 
Practitioners in the field are learning that most technologies are multifunctional and can 
be adapted to address a wide range of learning outcomes. Using a combination of tech- 
nologies is now being labeled the “third generation" of distance learning systems (the first 
and second being correspondence courses and the current use of single mediums, respec- 
tively). For instance, combining two-way interactive video with computer-mediated learning 
could prove to be a very powerful tool for learning, particularly higher order thinking 
skills. The third generation models often use a combination of print, videotape, audio 
tape, fax, audio conferencing, and voice mail. Combining these technologies with com- 
puter-mediated learning enhances interaction between student and student, and student 
and teacher. Unfortunately, there are few studies that examine more than one technol- 
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ogy— and the synergistic effects of certain technologies— in addressing specific educa- 
tional outcomes and particular student groups. The few studies that are available do not 
provide sufficient grounds for generalization because of a range of limitations, including 
small sample sizes and lack of sufficient explanation of the instructional treatment. 

The research does not include a theoretical or conceptual framework. 

There is a vital need to develop a more integrated, coherent, and sophisticated program of 
research on distance learning that is based on theory. Theories serve both as goals and 
tools— as means and ends. As goals, they provide explanations for specific phenomenon 
with maximal probability. As tools, they provide an underlying frame- 
work for observation and discovery by governing the kind of phe- 
nomena that investigators study. Theory allows researchers to build 
on the work of others and, increase the probability of addressing the 
more significant questions regarding distance learning. Using theory 
as a guiding framework also allows the research to be replicated and 
enhances its generalizability, therefore making individual studies 
more meaningful. Several authors have lamented that there are no theories that deal with 
the interactions and interrelationships of variables in terms of the effectiveness of distance 
learning programs. 

The research does not adequately address the effectiveness of digital "libraries." 

Students participating in distance learning, particularly those in remote locations, are 
often introduced to a digital "library" that provides access to bibliographies, as well as full- 
text, of a variety of resources. The library is at the core of the higher education experience 
and, especially at the graduate level, is an integral part of the teaching/learning process. 
Some digital libraries boast an enormous array of resources, with the implicit notion that 
they can provide the same service as the traditional library. Indeed, digital libraries asso- 
ciated with the University of California, Stanford University, and the University of Michi- 
gan have made great advances in providing access to a wealth of collections. 

There is some concern, however, that these showcase libraries are not representative of 
other digital libraries. For instance, is there a tendency for the digital library, because of a 
limited collection— specifically with respect to copyrighted materials— to adversely affect 
curricular decisions? Do digital libraries provide adequate services for the academic pro- 



Do digital libraries provide 
suffident and adequate services 
for the academic programs they 
are established to support? 
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grams they are established to support? Anecdotal evidence seems to suggest that the cur- 
riculum objectives of some distance learning courses have been altered because of a lim- 
ited variety of books and journals available from the digital library. 
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Implications 



R esearch on distance learning has been driven by what many are calling the "in* 
formation revolution.” The rapid evolution and advances in technology 
offer both the general public and faculty a dizzying array of challenges that are 
unprecedented. As Wulf stated in 1997 testimony before the U.S. Senate Committee on 
Labor and Human Resources, 

One of the hardest things for most people to understand is the effect of 
information technology's exponential rate of improvement. For the last four 
decades, the speed and storage capacity of computers have doubled every 
18-24 months; the cost, size, and power consumption have become smaller 
at about the same rate. The bandwidth of computer networks has increased 
a thousand-fold in just the last decade, and the traffic on the network con- 
tinues to grow at 300-500 percent annually. For the foreseeable future, all 
of these trends will continue; the basic technology to support their contin- 
ued advance exists now.^^ 

Ifechnology is having, and will continue to have, a profound impact on colleges and uni- 
versities in America and around the globe. Distance learning, which was once a poor and 
often unwelcome stepchild within the academic community, is becoming increasingly 
more visible as a part of the higher education family. But the research and literature 
reviewed for this paper indicate that the higher education community has a lot to learn 
regarding how, and in what ways, technology can enhance the teaching/learning process, 
particularly at a distance. 
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As with other educational innovations that have come before it, there is some danger that 
the innovations made possible through distance education are advancing more rapidly than 
our understanding of its practical uses. Princeton historian Robert Darnton makes this point 
in a recent essay about electronic publishing. Darnton observes that electronic publishing 
has passed through three stages since its inception in the 1940s: “an initial phase of utopian 
enthusiasm, a period of disillusionment, and a new tendency toward pragmatism. In the 
context of the research on distance learning and its effectiveness, more emphasis has been 
placed on the “utopian" possibilities of the technology and its potential to do as well as 
classroom-based instruction. But not enough “pragmatism" has been applied to allow for a 
discussion of distance learning's practical implications as a supplement to enhance teach- 
ing and learning. 

There are at least three broad implications that can be derived from this review of the 
original research and the other literature. The first is that the notion of “access to college" 
in the distance learning context is unclear. Many of the advocates of distance learning 
tout access to college-level education as a raison d'etre for the proliferation of distance 
education. Indeed, in some states, public policy leaders are recommending using distance 
education in lieu of “bricks and mortar" learning. Of particular concern is access as it 
relates to the efficacy of computer-mediated learning. Unlike two-way interactive video, 
where students and the instructor can see and talk to each other in a conventional class- 
room, computer-mediated learning requires special skills of students and more sophisti- 
cated technical support if students are to interact fully. 

Questions that need to be asked include the following: What is the "quality" of the access? 
Does the student have the necessary skills to use the technology? What are the best ways 



nology continues to advance, additional research is needed. Access is indeed a hollow 
concept if students experience insurmountable difficulties that they would not normally 
encounter on campus. 



The notion of "access to 
college" in the distance 
learning context is unclear. 



to participate in asynchronous communication? Is there adequate 
technical support? Perhaps most important, will the cost of pur- 
chasing a computer and maintaining software be prohibitive for a 
substantial number of students? Some case studies and descrip- 
tive research have addressed these issues; however, as the tech- 




Implications 




Second, it seems clear that technology cannot replace the human factor in higher edu- 
cation. As William Massy notes, "The faculty role will change from being mainly a con- 
tent expert ('the professor's job is to profess') to a combination of 

content expert, learning process design expert, and process imple- It S66mS Cl6ar that technology 

mentation manager. Faculty also will be motivators and mentors, cannOt replace the human factor 
interpreters (especially of non-codified knowledge), and, as a col- higher education. 

league recently put it, 'expert learners'— people who lead the learn- 

ing process by breaking the trail and setting the right personal example. Technology can 
leverage faculty time, but it cannot replace most human contact without significant 
quality losses."^"* 

Third, although the ostensible purpose of much of the research is to ascertain how tech- 
nology affects student learning and student satisfaction, many of the results seem to indi- 
cate that technology is not nearly as important as other factors, such as learning tasks, 
learner characteristics, student motivation, and the instructor. The irony is that the bulk 
of the research on technology ends up addressing an activity that is fundamental to the 
academy, namely pedagogy— the art of teaching. Tb that extent, the research has had a 
salutary effect in that a rising tide lifts all boats. Any discussion about enhancing the 
teaching/learning process through technology also has the beneficial effect of improving 
how students are taught on campus. 

Consider this example. In 1987 the American Association of Higher Education (AAHE) 
published "Seven Principles for Good Practice in Undergraduate Education," which dis- 
tilled findings from the research on the undergraduate experience. The principles were 
revived in 1996 to enable those using new communication and information technologies 
to enhance the teaching/learning process. In one form or another, the principles have 
been incorporated in a variety of publications on good practice, and were replete in many 
of the studies.^^ 
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AAHE's Principles for Good Practice in Undergraduate Education 
include those methods that: 

► encourage contacts between students and faculty; 

► develop reciprocity and cooperation among students; 

► use active learning techniques; 

► give prompt feedback; 

► emphasize time-on-task; 

► communicate high expectations; and 

► respect diverse talents and ways of learning. 



In a sense, the discussion has come full circle. As shown in this paper, the research on 
distance learning has a long way to go, and much of it is inconclusive. On the other hand, 
technology has helped the academy to continue its focus on the essential goals of teach- 
ing and learning. As a result, either implicitly or explicitly, the key question that needs to 
be asked is: What is the best way to teach students? 



ERIC 



39 




APPENDIX A 

Overview of the Original Research 



The following descriptions of the original research on the effectiveness of learning con- 
vey a general understanding of the range and variety of studies undertaken. This over- 
view is intended to provide abroad understanding of what the research has examined, the 
research methodologies used, the academic disciplines studied, and the types of institu- 
tions where the research is occurring. 

What Technologies Have Been Studied in Distance Learning Research? 

One simple way to get a sense of the research on learning is to determine which types of 
technology are being studied. As noted earlier, distance learning can involve a number of 
technologies, ranging from two-way interactive video, to two-way audio, to web-based 
asynchronous communication. As shown in Fig- 
ure One, about one-quarter of the research stud- 
ies that were analyzed focused on two-way in- 
teractive video, and another quarter targeted 
computer-mediated learning.^® Lesser percent- 
ages addressed one-way broadcast communica- 
tion, and one-way prerecorded video. The re- 
maining research articles studied other combi- 
nations of technologies. 

What Type of Research Designs Are Used to 
Study Distance Learning? 

It is also instructive to understand the types of 
research designs being used in the study of dis- 



Figure One: Types of Distance Learning 
Technology Studied 



Other (9%) 
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tance learning. As shown in Figure Two, more 
than four out of five of the research designs were 
either experimental research or descriptive re- 
search. Case studies and correlational studies 
represented only a small proportion of the total. 

What Academic Disciplines Are Being Studied? 

There is some debate regarding the academic 
disciplines that are appropriate for distance 
learning. For instance, given the technology that 
is available today, it would be difficult to con- 
duct a “public speaking" course using computer- 
mediated learning. However, the same course 
could be conducted using two-way interactive 
video. The more vocal proponents of distance learning insist that virtually any subject 
matter can be communicated through electronic means, emphasizing the accelerating 
pace of technology enhancements. 

There appears to be a wide variety of academic disciplines under study. Of the studies 
reviewed, three-quarters focused on undergraduate courses. These data are consistent 
with the survey results from NCES, where over 60 percent of the distance education courses 
were undergraduate.^^ At the undergraduate level, traditional liberal arts courses (humani- 
ties, social sciences, and math/science) represented almost two-thirds of the disciplines 
examined. Business and computer science courses represented most of the remaining 
studies that were reviewed. 



Figure Two: Characterizing Distance 
Learning Research Studies by Research 
Methods 




Of the relatively few studies that examined graduate courses, there is much less variety. 
In this group, almost half involved education courses. The remainder included social 
sciences and, to a lesser extent, nursing, business, and math/ science. 

Where Is the Research Being Conducted? 

The vast majority of original research studies was conducted at four-year institutions. 
Moreover, at these four-year institutions, virtually every type of technology was under 
study, with two-way interactive video and computer-mediated learning receiving the most 
emphasis. Of the studies that were reviewed, only two-way interactive video, computer- 
mediated learning, and one-way broadcast were studied at two-year institutions. 




APPENDIX B 

Other Articles, Essays, 
and Research Syntheses 



I n addition to analyzing original research on the effectiveness of learning, this re- 
view also examined articles in major distance learning publications to ascertain the 
nature of the subjects discussed since 1990. The sources reviewed included: The 
Journal of Research on Computing in Education, The American Journal of Distance Education, 
The Journal of Distance Education, The Journal of Computer-Based Instruction, Research in 
Distance Education, publications produced by the University of Maryland Institute for Dis- 
tance Education, publications produced by the American Center for the Study of Distance 
Education, and Ed: The Official Publication of the U.S. Distance Learning Association. 

This review of several hundred articles and essays revealed that course and program 
design for distance learning commanded the most attention in the literature. A significant 
number of articles addressed issues related to the operational and administrative aspects 
of providing distance education activities. In general, the articles provided "how to" es- 
says for a variety of technologies and content areas. Many were institution-specific and 
attempted to identify principles of good practice. 

Another prevalent topic in the literature related to the effectiveness of distance educa- 
tion. A number of articles addressed how technologies affected learner outcomes, student 
satisfaction, and student attitudes toward learning at a distance. The majority of articles 
provided essays or descriptions regarding the effectiveness of particular technologies at 
specific institutions. Others identified important variables such as collaboration, 
interactivity, and learning styles. 

A significant number of articles revolved around general research that asked questions 
such as: What is distance learning? What is the role of distance learning in higher educa- 
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tion? How can distance learning address access and quality? Issues like lifelong learning 
and the notion of learning anytime and anyplace were explored in relation to distance 
learning. Several of the articles reviewed past research and provided perspectives from 
the experiences of individual institutions. 

These three areas— course and program design, effectiveness of technology, and general 
research— comprised the majority of articles in these mainstream distance learning pub- 
lications. However, student support received considerable attention in many articles. Sub- 
jects such as advising and counseling, student records, and financial aid were explored. 
Commentaries also were found in several journals relating to existing technologies and 
how they can be better utilized, as well as the potential for emerging technologies. 

Subject areas that received the least attention in this review of major distance learning 
publications were faculty issues and library services. The articles on faculty addressed 
concerns such as teaching workload, professional development, and technical support for 
faculty engaged in distance learning. Library issues related to the quality and accessibility 
of digital libraries. 
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